Expander Graph and Communication-Efficient Decentralized Optimization

In this paper, we discuss how to design the graph topology to reduce the communication complexity of certain algorithms for decentralized optimization. Our goal is to minimize the total communication needed to achieve a prescribed accuracy. We discover that the so-called expander graphs are near-optimal choices. We propose three approaches to construct expander graphs for different numbers of nodes and node degrees. Our numerical results show that the performance of decentralized optimization is significantly better on expander graphs than other regular graphs.Comment: 2016 IEEE Asilomar Conference on Signals, Systems, and Computer

Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model

In the coming years and decades, autonomous vehicles (AVs) will become increasingly prevalent, offering new opportunities for safer and more convenient travel and potentially smarter traffic control methods exploiting automation and connectivity. Car following is a prime function in autonomous driving. Car following based on reinforcement learning has received attention in recent years with the goal of learning and achieving performance levels comparable to humans. However, most existing RL methods model car following as a unilateral problem, sensing only the vehicle ahead. Recent literature, however, Wang and Horn [16] has shown that bilateral car following that considers the vehicle ahead and the vehicle behind exhibits better system stability. In this paper we hypothesize that this bilateral car following can be learned using RL, while learning other goals such as efficiency maximisation, jerk minimization, and safety rewards leading to a learned model that outperforms human driving. We propose and introduce a Deep Reinforcement Learning (DRL) framework for car following control by integrating bilateral information into both state and reward function based on the bilateral control model (BCM) for car following control. Furthermore, we use a decentralized multi-agent reinforcement learning framework to generate the corresponding control action for each agent. Our simulation results demonstrate that our learned policy is better than the human driving policy in terms of (a) inter-vehicle headways, (b) average speed, (c) jerk, (d) Time to Collision (TTC) and (e) string stability

Degradation Mechanisms and Mitigation Strategies of Nickel-Rich NMC-Based Lithium-Ion Batteries

Abstract The demand for lithium-ion batteries (LIBs) with high mass-specific capacities, high rate capabilities and long-term cyclabilities is driving the research and development of LIBs with nickel-rich NMC (LiNixMnyCo1−x−yO2, $$x \geqslant 0.5$$x⩾0.5) cathodes and graphite (LixC6) anodes. Based on this, this review will summarize recently reported and widely recognized studies of the degradation mechanisms of Ni-rich NMC cathodes and graphite anodes. And with a broad collection of proposed mechanisms on both atomic and micrometer scales, this review can supplement previous degradation studies of Ni-rich NMC batteries. In addition, this review will categorize advanced mitigation strategies for both electrodes based on different modifications in which Ni-rich NMC cathode improvement strategies involve dopants, gradient layers, surface coatings, carbon matrixes and advanced synthesis methods, whereas graphite anode improvement strategies involve surface coatings, charge/discharge protocols and electrolyte volume estimations. Electrolyte components that can facilitate the stabilization of anodic solid electrolyte interfaces are also reviewed, and trade-offs between modification techniques as well as controversies are discussed for a deeper understanding of the mitigation strategies of Ni-rich NMC/graphite LIBs. Furthermore, this review will present various physical and electrochemical diagnostic tools that are vital in the elucidation of degradation mechanisms during operation to supplement future degradation studies. Finally, this review will summarize current research focuses and propose future research directions. Graphic Abstract The demand for lithium-ion batteries (LIBs) with high mass specific capacities, high rate capabilities and longterm cyclabilities is driving the research and development of LIBs with nickel-rich NMC (LiNixMnyCo1−x−yO2, x ≥ 0.5) cathodes and graphite (LixC6) anodes. Based on this, this review will summarize recently reported and widely recognized studies of the degradation mechanisms of Ni-rich NMC cathodes and graphite anodes. And with a broad collection of proposed mechanisms on both atomic and micrometer scales, this review can supplement previous degradation studies of Ni-rich NMC batteries. In addition, this review will categorize advanced mitigation strategies for both electrodes based on different modifications in which Ni-rich NMC cathode improvement strategies involve dopants, gradient layers, surface coatings, carbon matrixes and advanced synthesis methods, whereas graphite anode improvement strategies involve surface coatings, charge/discharge protocols and electrolyte volume estimations. Electrolyte components that can facilitate the stabilization of anodic solid-electrolyte interfaces (SEIs) are also reviewed and tradeoffs between modification techniques as well as controversies are discussed for a deeper understanding of the mitigation strategies of Ni-rich NMC/graphite LIBs. Furthermore, this review will present various physical and electrochemical diagnostic tools that are vital in the elucidation of degradation mechanisms during operation to supplement future degradation studies. Finally, this review will summarize current research focuses and propose future research directions

Learning Meta Model for Zero- and Few-shot Face Anti-spoofing

Face anti-spoofing is crucial to the security of face recognition systems. Most previous methods formulate face anti-spoofing as a supervised learning problem to detect various predefined presentation attacks, which need large scale training data to cover as many attacks as possible. However, the trained model is easy to overfit several common attacks and is still vulnerable to unseen attacks. To overcome this challenge, the detector should: 1) learn discriminative features that can generalize to unseen spoofing types from predefined presentation attacks; 2) quickly adapt to new spoofing types by learning from both the predefined attacks and a few examples of the new spoofing types. Therefore, we define face anti-spoofing as a zero- and few-shot learning problem. In this paper, we propose a novel Adaptive Inner-update Meta Face Anti-Spoofing (AIM-FAS) method to tackle this problem through meta-learning. Specifically, AIM-FAS trains a meta-learner focusing on the task of detecting unseen spoofing types by learning from predefined living and spoofing faces and a few examples of new attacks. To assess the proposed approach, we propose several benchmarks for zero- and few-shot FAS. Experiments show its superior performances on the presented benchmarks to existing methods in existing zero-shot FAS protocols.Comment: Accepted by AAAI202

Improving the generalizability and robustness of large-scale traffic signal control

A number of deep reinforcement-learning (RL) approaches propose to control traffic signals. In this work, we study the robustness of such methods along two axes. First, sensor failures and GPS occlusions create missing-data challenges and we show that recent methods remain brittle in the face of these missing data. Second, we provide a more systematic study of the generalization ability of RL methods to new networks with different traffic regimes. Again, we identify the limitations of recent approaches. We then propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble. Building upon the state-of-the-art previous model which uses a decentralized approach for large-scale traffic signal control with graph convolutional networks (GCNs), we first learn models using a distributional reinforcement learning (DisRL) approach. In particular, we use implicit quantile networks (IQN) to model the state-action return distribution with quantile regression. For traffic signal control problems, an ensemble of standard RL and DisRL yields superior performance across different scenarios, including different levels of missing sensor data and traffic flow patterns. Furthermore, the learning scheme of the resulting model can improve zero-shot transferability to different road network structures, including both synthetic networks and real-world networks (e.g., Luxembourg, Manhattan). We conduct extensive experiments to compare our approach to multi-agent reinforcement learning and traditional transportation approaches. Results show that the proposed method improves robustness and generalizability in the face of missing data, varying road networks, and traffic flows